Solving complicated AI tasks with different domains and modalities is a key step toward artificial general intelligence. While there are numerous AI models available for various domains and modalities, they cannot handle complicated AI tasks autonomously. Considering large language models (LLMs) have exhibited exceptional abilities in language understanding, generation, interaction, and reasoning, we advocate that LLMs could act as a controller to manage existing AI models to solve complicated AI tasks, with language serving as a generic interface to empower this. Based on this philosophy, we present HuggingGPT, an LLM-powered agent that leverages LLMs (e.g., ChatGPT) to connect various AI models in machine learning communities (e.g., Hugging Face) to solve AI tasks. Specifically, we use ChatGPT to conduct task planning when receiving a user request, select models according to their function descriptions available in Hugging Face, execute each subtask with the selected AI model, and summarize the response according to the execution results. By leveraging the strong language capability of ChatGPT and abundant AI models in Hugging Face, HuggingGPT can tackle a wide range of sophisticated AI tasks spanning different modalities and domains and achieve impressive results in language, vision, speech, and other challenging tasks, which paves a new way towards the realization of artificial general intelligence.

Hardware-specific optimizations in machine learning (ML) frameworks can cause numerical deviations of inference results. Quite surprisingly, despite using a fixed trained model and fixed input data, inference results are not consistent across platforms, and sometimes not even deterministic on the same platform. We study the causes of these numerical deviations for convolutional neural networks (CNN) on realistic end-to-end inference pipelines and in isolated experiments. Results from 75 distinct platforms suggest that the main causes of deviations on CPUs are differences in SIMD use, and the selection of convolution algorithms at runtime on GPUs. We link the causes and propagation effects to properties of the ML model and evaluate potential mitigations. We make our research code publicly available.

Here we introduce methods to extend the approach to models with higher-order potentials and develop theoretical insights. In particular, we show that the triplet-consistent polytope TRI is unique in being'universally rooted'.

Low-dose computed tomography (CT) represents a significant improvement in patient safety through lower radiation doses, but increased noise, blur, and contrast loss can diminish diagnostic quality. Therefore, consistency and robustness in image quality assessment become essential for clinical applications. In this study, we propose an LLM-based quality assessment system that generates both numerical scores and textual descriptions of degradations such as noise, blur, and contrast loss. Furthermore, various inference strategies - from the zero-shot approach to metadata integration and error feedback - are systematically examined, demonstrating the progressive contribution of each method to overall performance. The resultant assessments yield not only highly correlated scores but also interpretable output, thereby adding value to clinical workflows. The source codes of our study are available at https://github.com/itu-biai/lmms_ldct_iqa.

The rise of data-driven algorithms and, most notably, applications of deep learning has led to concerns about a lack of thorough human oversight of socially important decisions that are either delegated in their entirety to machines, or made by humans based on machine recommendations. Explainable AI (XAI) approaches attempt to mitigate these concerns by helping (typically human) users understand how and why algorithms produce specific outputs [1]. An important class of XAI methods focuses on providing explanations of black-box classifiers that attribute classification outcomes (which one may consider decisions or decision recommendations) to input characteristics (feature values) [2, 3]. Such feature attribution methods can be considered meta-reasoning functions that approximate classifier behavior with the objective of providing users a reasonably faithful intuition of behavioral fundamentals. One of the most prominent feature attribution methods is SHAP, which is based on the Shapley value in cooperative game theory that quantifies players' (feature values') contributions to coalition utility (classification outcomes) [4]. Feature attribution methods have, in general, limitations: notably, they are necessarily approximations, and as purely technical tools, they cannot fully consider crucial nuances of the socio-technical systems they are embedded in [5]; for example, the visualizations provided out-of-the-box by feature attribution software libraries may be difficult to interpret [6]. Still, Shapley value-based approaches can be considered a reasonable choice for facilitating black-box explainability, notably because (i) they are based on well-established and intuitive mathematical principles of the Shapley value and (ii) there is at least some evidence of their potential usefulness, also relative to alternative approaches [6]. However, the Shapley value cannot straight-forwardly be applied to inference from Large Language Models (LLMs), which power many of the currently emerging AI applications.

EdgeSync: Accelerating Edge-Model Updates for Data Drift through Adaptive Continuous Learning Runchu Dong, Peng Zhao, Guiqin Wang, Nan Qi, Jie Lin A more efficient edge-model updating approach that automatically and continuously adapts models to the scene with data drift. A novel method for filtering video streaming samples that integrates timeliness and adaptability to eliminate unnecessary samples. A continuous training manager that optimizes the training schedule and duration using both labeled and computed features. Abstract Real-time video analytics systems typically deploy lightweight models on edge devices to reduce latency. However, the distribution of data features may change over time due to various factors such as changing lighting and weather conditions, leading to decreased model accuracy. Recent frameworks try to address this issue by leveraging remote servers to continuously train and adapt lightweight edge models using more complex models in the cloud. Despite these advancements, existing methods face two key challenges: first, the retraining process is compute-intensive, causing significant delays in model updates; second, the new model may not align well with the evolving data distribution of the current video stream. To address these challenges, we introduce EdgeSync, an efficient edge-model updating approach that enhances sample filtering by incorporating timeliness and inference results, thus ensuring training samples are more relevant to the current video content while reducing update delays. Additionally, EdgeSync features a dynamic training management module that optimizes the timing and sequencing of model updates to improve their timeliness. Evaluations on diverse and complex real-world datasets demonstrate that EdgeSync improves accuracy by approximately 3.4% compared to existing methods and by about 10% compared to traditional approaches. Introduction Real-time video analytics has significant potential across a range of applications, including augmented reality, video surveillance, and traffic detection [1]. Recent advancements in deep neural networks (DNNs) have significantly improved the performance of video analysis, with some models even surpassing human accuracy in certain scenarios [2, 3, 4].

In the design of customer relationship management (CRM) systems, accurately identifying customer types and offering personalized services are key to enhancing customer satisfaction and loyalty. However, this process faces the challenge of discerning customer voices and intentions, and general pre-trained automatic speech recognition (ASR) models make it difficult to effectively address industry-specific speech recognition tasks. To address this issue, we innovatively proposed a solution for fine-tuning industry-specific ASR models, which significantly improved the performance of the fine-tuned ASR models in industry applications. Experimental results show that our method substantially improves the crucial auxiliary role of the ASR model in industry CRM systems, and this approach has also been adopted in actual industrial applications.

Leveraging advancements in artificial intelligence, we introduce a deep learning-based method for efficiently characterizing heterostructures and 2D materials, specifically MoS 2-MoSe 2 lateral heterostructures and MoS 2 flakes with varying shapes and thicknesses. By utilizing YOLO models, we achieve an accuracy rate of over 94.67% in identifying these materials. Additionally, we explore the application of transfer learning across different materials, which further enhances model performance. This model exhibits robust generalization and anti-interference ability, ensuring reliable results in diverse scenarios. To facilitate practical use, we have developed an application that enables real-time analysis directly from optical microscope images, making the process significantly faster and more cost-effective than traditional methods. This deep learning-driven approach represents a promising tool for the rapid and accurate characterization of 2D materials, opening new avenues for research and development in material science. Keywords 2D material, TMDs, lateral heterostructure, deep learning, instance segmentation, morphology characterization Introduction Two-dimensional (2D) materials have attracted significant attention due to their excellent mechanical, electrical, thermal, and optical properties, making them ideal candidates for next-generation technologies.